CLAIMS 



What is claimed is: 

1 . A method for coding macroblock video data according to layered coding 
techniques in which the macroblock video data is represented as multi-layered frames, each 
frame having a plurality of references in multiple layers ranging from a base layer of low 
quality to enhancement layers of increasingly higher quality, wherein the method uses a 
coding mode to select a reference to be used for prediction and reconstruction at each 
macroblock of video data, the method comprising the steps of: 

encoding macroblocks to produce a first bitstream representing a base layer; 
encoding macroblocks to produce a second bitstream representing one or more 
enhancement layers using an INTER coding mode selected from the group consisting of: 
an LPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a low quality reference in a previous frame, wherein a 
high quality reference of the current frame is reconstructed from the low quality 
reference in the previous frame; 

an HPHR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein 
the high quality reference in the previous frame is used to reconstruct a high quality 
reference in the current frame; and 

an HPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 

a high quality reference of the current frame is reconstructed 
from a low quality reference in the previous frame; and 
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the high quality reference in the previous frame is not the same as 
the low quality reference in the previous frame. 

2. The method as set forth in Claim 1, wherein: 

encoding macroblocks to produce the first and second bitstreams further comprises 
estimating drifting information that occurs from the encoding; and 

the INTER coding mode is selected from the group based upon a derivation that uses 
the estimated drifting information to maximize coding efficiency while minimizing drifting 
error. 

3. The method as set forth in Claim 1 , wherein: 

encoding macroblocks to produce a first bitstream representing a base layer 
comprises motion compensating an original image using the base layer as a reference to 
form a low quality predicted image in the pixel domain Xb; 

encoding macroblocks to produce a second bitstream representing one or more 
enhancement layers using an INTER coding mode further comprises motion compensating 
an original image using the enhancement layer as a reference to form a high quality 
predicted image in the pixel domain Xe; 

transforming the low quality predicted image in the pixel domain Xb to form low 
quality predicted coefficients Xb; 

quantizing the low quality predicted coefficients to form quantized coefficients; 
dequantizing the quantized coefficients to form dequantized coefficients Jf^ ; 
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inverse transforming the dequantized coefficients to form inverse dequantized 

coefficients as a reconstructed base layer in pixel the domain ; 

taking the absolute value of the difference between 3c^ and: 

the base layer of the motion compensated original image in the pixel domain 
Xb to form a first value; and 

the high quality predicted image in the pixel domain Xe to form a second 

value; 

selecting the LPLR mode when the first value less than or equal to the second value; 

and 

Q selecting the HPLR mode or the HPHR mode when the first value greater then the 

D 

O second value. 

SJ 
O 
SI 

^ 4. A method as recited in Claim 1 , further comprising: 

U using variable length coding to compress the first and second bitstreams; 

^ transmittinjg the compressed first and second bitstreams over a network; 

Q 

M decompressing and decoding the first bitstream representing the base layer into the 

video data; and 

decompressing and decoding the second bitstream representing one or more 
enhancement layers into the video data. 



5. A method as recited in Claim 4, further comprising reconstructing a missing 
enhancement layer from one of the high and low high quality references in the previous 
frame. 
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6. The method as defined in Claim 1, wherein the layered coding techniques 
comprise a Progressive Fine Granularity Scalable (PFGS) video coding. 

7. A method for coding macroblock video data according to layered coding 
techniques in which the macroblock video data is represented as multi-layered frames, each 
frame having a plurality of references in multiple layers ranging from a base layer of low 
quality to enhancement layers of increasingly higher quality, the method comprising the 
steps of: 

motion compensating an original image using the base layer as a reference to form a 
low quality predicted image; 

transforming the low quality predicted image to form low quality predicted 
coefficients; 

quantizing the low quality predicted coefficients to form quantized coefficients; 
dequantizing the quantized coefficients to form dequantized coefficients; 
inverse transforming the dequantized coefficients to form inverse dequantized 
coefficients in the form of a reconstructed base layer in pixel the domain ; 

taking the absolute value of the difference between and: 

the base layer of the motion compensated original image in the pixel domain 
Xb to form a first value; and 

one said enhancement layer of the motion compensated original image in the 
pixel domain Xe to form a second value; 
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selecting the LPLR mode when the first value less than or equal to the second value; 

and 

selecting the HPLR mode or the HPHR mode when the first value greater then the 
second value. 

8. The method as defined in Claim 7, wherein the layered coding techniques 
comprise a Progressive Fine Granularity Scalable (PFGS) video coding. 

9. A computer-readable medium having computer-executable instructions, 
which when executed on a processor, direct a computer to perform the steps of Claim 7. 

1 0. A computer-readable medium having computer-executable instructions, 
which when executed on a processor, direct a computer to: 

encode macroblocks of video data according to layered coding techniques in which the 
macroblock video data is represented as multi-layered frames, each frame having a plurality 
of references in multiple layers ranging from a base layer of low quality to enhancement 
layers of increasingly higher quality, including encoding macroblocks to produce a first 
bitstream representing a base layer, and encoding macroblocks to produce a second 
bitstream representing one or more enhancement layers using an INTER coding mode 
selected from the group consisting of: 

an LPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a low quality reference in a previous fi-ame, wherein a 
high quality reference of the current frame is reconstructed from the low quality 
reference in the previous frame; 
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an HPHR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein 
the high quality reference in the previous frame is used to reconstruct a high quality 
reference in the current frame; and 

an HPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a high quality reference of the current frame is reconstructed from a low 

quality reference in the previous frame; and 

the high quality reference in the previous frame is not the same as the low 

quality reference in the previous frame. 

11. A computer-readable medium as recited in claim 10, further having 
instructions that direct a computer to store the base layer and the one or more enhancement 
layers in memory. 

12. A computer-readable medium as recited in claim 10, fixrther having 
instructions that direct a computer to: 

transmit the base layer over a network; and 

transmit the one or more enhancement layers over the network according to 
bandwidth availability on the network. 

13. A computer-readable medium as recited in claim 10, fiirther having 
instructions that direct a computer to recover the video data from the base layer and any of 
the one or more enhancement layers. 
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14. A computer-readable medium as recited in claim 10, further having 
instructions that direct a computer to reconstruct a missing enhancement layer from an 
enhancement layer of a reference reconstructed frame. 

15. A video coding system to encode macroblocks of video data according to 
layered coding techniques in which the macroblock video data is represented as multi- 
layered frames, each frame having a plurality of references in multiple layers ranging from a 
base layer of low quality to enhancement layers of increasingly higher quality, the video 

^ coding system comprising: 

p a base layer encoder to encode macroblocks to produce a first bitstream representing a 

^ base layer; and 

^ an enhancement layer encoder to encode macroblocks to produce a second bitstream 

a 

representing one or more enhancement layers using an INTER coding mode selected from 
O the group consisting of: 

5 

an LPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a low quality reference in a previous frame, wherein a 
high quality reference of the current frame is reconstructed from the low quality 
reference in the previous frame; 

an HPHR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein 
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the high quality reference in the previous frame is used to reconstruct a high quality 
reference in the current frame; and 

an HPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a high quality reference of the current frame is reconstructed from a 
low quality reference in the previous frame; and 

the high quality reference in the previous frame is not the same as the low 
quality reference in the previous frame. 



o 
O 
□ 

p 16. An operating system comprising the video coding system of claim 15. 

. SI 

P 

m 

Q 
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17. A video coding system for encoding macrobiocks of video data according to 
layered coding techniques in which the macroblock video data is represented as multi- 
layered frames, each frame having a plurality of references in multiple layers ranging from a 
base layer of low quality to enhancement layers of increasingly higher quality, the video 
coding system comprising: 

means for encoding macrobiocks to produce a first bitstream representing a base 

layer; and 

means for encoding macrobiocks to produce a second bitstream representing one 
or more enhancement layers using an INTER coding mode selected from the group 
consisting of: ^ 

an LPLR coding mode that encodes macrobiocks to produce the second 
bitstream by a prediction from a low quality reference in a previous frame, wherein a 
high quality reference of the current frame is reconstructed from the low quality 
reference in the previous frame; 

an HPHR coding mode that encodes macrobiocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein 
the high quaHty reference in the previous frame is used to reconstruct a high quality 
reference in the current frEune; and 

an HPLR coding mode that encodes macrobiocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a high quality reference of the current frame is reconstructed from a 

low quality reference in the previous frame; and 

the high quality reference in the previous firame is not the same as the low 

quality reference in the previous frame. 
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18. An operating system comprising the video coding system of claim 1 7. 
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19. A video delivery architecture, comprising: 

a content provider having a video storage to store video data, a video server to serve 
the video data over a network, a base layer encoder, and an enhancement layer encoder, the 
video server being configured to encode macrobiocks of video data according to layered 
coding techniques in which the macroblock video data is represented as multi-layered 
frames, each frame having a plurality of references in multiple layers ranging from a base 
layer of low quality to enhancement layers of increasingly higher quality, wherein: 
the base layer encoder encodes macrobiocks to produce a first bitstream 
representing a base layer; and 

the enhancement layer encoder encodes macrobiocks to produce a second 
bitstream representing one or more enhancement layers using an INTER coding 
mode selected from the group consisting of: 

an LPLR coding mode that encodes macrobiocks to produce the 
second bitstream by a prediction from a low quality reference in a previous 
frame, wherein a high quality reference of the current frame is reconstructed 
from the low quality reference in the previous frame; 

an HPHR coding mode that encodes macrobiocks to produce the 
second bitstream by a prediction from a high quality reference in a previous 
frame, wherein the high quality reference in the previous frame is used to 
reconstruct a high quality reference in the current frame; and 

an HPLR coding mode that encodes macrobiocks to produce the 
second bitstream by a prediction from a high quality reference in a previous 
frame, wherein: 
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a high quality reference of the current frame is reconstructed 
from a low quality reference in the previous frame; and 

the high quality reference in the previous frame is not the same as 
the low quality reference in the previous frame; 
a client configured to receive the encoded video data served from the content 
provider, the client being configured to decode the video data. 

20. A video delivery architecture as recited in claim 19, wherein: 

the video server transmits the encoded video data as composing the base layer and the 
one or more of the enhancement layers; and 

the client decodes the video data from the base layer and the one or more enhancement 
layers. 

21 . A video delivery architecture as recited in claim 19, wherein the client 
reconstructs an enhancement layer in a particular frame from an enhancement layer of a 
reference reconstructed frame. 
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22. A method for coding macroblock video data according to layered coding 
techniques in which the macroblock video data is represented as multi-layered frames, each 
frame having a plurality of references in multiple layers ranging from a base layer of low 
quality to enhancement layers of increasingly higher quality, wherein the method uses a 
coding mode to select a reference to be used for prediction and reconstruction both at the 
base layer and at the enhancement layer, the method comprising: 

encoding macroblocks to produce a first bitstream representing a base layer; 
encoding macroblocks to produce a second bitstream representing one or more 
enhancement layers using an INTER coding mode selected from the group consisting of: 
a BHPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a low quality reference of the current frame is reconstructed from, and 
is of the same quality as, a low quality reference in the previous frame; and 
a high quality reference of the current frame is reconstructed from, 
and is of the same quality as, the high quality reference in the previous frame; 
a BHPHR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a low quality reference of the current frame is reconstructed from the 
high quality reference in the previous frame; and 

a high quality reference of the current frame is reconstructed from, 
and is of the same quality as, the high quality reference in the previous frame. 

23. A computer-readable medium having computer-executable instructions, 
which when executed on a processor, direct a computer to perform the steps of Claim 22. 
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24. The method as defined in Claim 22, wherein the layered coding techniques 
comprise a Progressive Fine Granularity Scalable (PFGS) video coding. 
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25. A video delivery architecture, comprising: 

a content provider having a video storage to store video data, a video server to serve 
the video data over a network, a base layer encoder, and an enhancement layer encoder, the 
video server being configured to encode macroblocks of video data according to layered 
coding techniques in which the macroblock video data is represented as multi-layered 
frames, each frame having a plurality of references in multiple layers ranging from a base 
layer of low quality to enhancement layers of increasingly higher quality, wherein: 
the base layer encoder encodes macroblocks to produce a first bitstream 
representing a base layer; and 

the enhancement layer encoder encodes macroblocks to produce a second 
bitstream representing one or more enhancement layers using an INTER coding 
mode selected from the group consisting of: 

a BHPLR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a low quality reference of the current frame is reconstructed from, and 
is of the same quality as, a low quality reference in the previous frame; and 
a high quality reference of the current frame is reconstructed from, 
and is of the same quality as, the high quality reference in the previous frame; 
a BHPHR coding mode that encodes macroblocks to produce the second 
bitstream by a prediction from a high quality reference in a previous frame, wherein: 
a low quality reference of the current frame is reconstructed from the 
high quality reference in the previous frame; and 
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a high quality reference of the current frame is reconstructed from, 



and is of the same quality as, the high quality reference in the 



previous frame; 



a client configured to receive the encoded video data served from the content 
provider, the client being configured to decode the video data. 

26. A video delivery architecture as recited in claim 25, wherein: 

the video server transmits the encoded video data as composing the base layer and the 
one or more of the enhancement layers; and 

the client decodes the video data from the base layer and the one or more enhancement 
layers. 

27. A video delivery architecture as recited in claim 25, wherein the client 
reconstructs an enhancement layer in a particular frame from an enhancement layer of a 
reference reconstructed frame. 
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28. A method for coding macroblock video data according to layered coding 
techniques in which the macroblock video data is represented as multi-layered frames in 
multiple layers ranging from a base layer of low quality to enhancement layers of 
increasingly higher quality, the frames including frames n-1, n, wherein frames n-I 

and have a plurality of references, and wherein the method uses a coding mode to select 
a reference to be used for prediction and reconstruction both in a backward direction in the 
frames and in a forward direction in the frames, the method comprising: 

encoding macroblocks to produce a first bitstream representing a base layer; 
encoding macroblocks to produce a second bitstream representing one or more 
^ enhancement layers using an INTER coding mode selected from the group consisting of: 

a first coding mode that encodes macroblocks to produce the second 



P 



P 



5 

1= 



bitstream for the n frame by temporal predictions from both a low quality reference 
O in the n-I frame and a low quality reference in the n+1 frame; 

m 

P a second coding mode that encodes macroblocks to produce the second 

bitstream for the n frame by temporal predictions from: 
a low quality reference in the n-I frame; 
a low quality reference in the n-^J frame; 
a high quality reference in the n-I frame; and 
a high quality reference in the n-^I frame; 
wherein the low quality references in the n-l and a7+7 frames are used to 
predict only the lowest quality enhancement layer in the n frame; 
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a third coding mode that encodes macroblocks to produce the second 
bitstream for the n frame by temporal predictions from: 

a high quality reference in the n-1 frame; and 
a high quality reference in the «+7 frame. 

29. A computer-readable medium having computer-executable instructions, 
which when executed on a processor, direct a computer to perform the steps of Claim 28. 



30. The method as defined in Claim 28, wherein the layered coding techniques 
comprise a Progressive Fine Granularity Scalable (PFGS) video coding. 

P 
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31. A video delivery architecture, comprising: 

a content provider having a video storage to store video data, a video server to serve 
the video data over a network, a base layer encoder, and an enhancement layer encoder, the 
video server being configured to encode macroblocks of video data according to layered 
coding techniques in which the macroblock video data is represented as multi-layered 
frames, each frame having a plurality of references in multiple layers ranging from a base 
layer of low quality to enhancement layers of increasingly higher quality, the frames 
including frames and n-^l, wherein frames n-I and «+7 have a plurality of 

references, wherein: 



the base layer encoder encodes macroblocks to produce a first bitstream 



P 
Q 

Q 



representing a base layer; and 



bitstream representing one or more enhancement layers using an INTER coding 



the enhancement layer encoder encodes macroblocks to produce a second 



Q 
O 



bitstream for the n frame by temporal predictions from both a low quality reference 



mode selected from the group consisting of: 



a first coding mode that encodes macroblocks to produce the second , 



in the n-I frame and a low. quality reference in the «+7 frame; 



a second coding mode that encodes macroblocks to produce the second 



bitstream for the n frame by temporal predictions from: 



a low quality reference in the n-J frame; 



a low quality reference in the frame; 



a high quality reference in the n-1 frame; and 



a high quality reference in the «+7 frame; 
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wherein the low quaUty references in the n-l and n^l frames are used to 
predict only the lowest quality enhancement layer in the n frame; 

a third coding mode that encodes macroblocks to produce the second 
bitstream for the n frame by temporal predictions from: 

a high quality reference in the n-l frame; and 
a high quality reference in the frame; 
a client configured to receive the encoded video data served from the content 
provider, the client being configured to decode the video data. 



1= 32. A video delivery architecture as recited in claim 3 1 , wherein: 

O 

p the video server transmits the encoded video data as composing the base layer and the 

O one or more of the enhancement layers; and 

M 

=P the client decodes the video data from the base layer and the one or more enhancement 



O 



layers. 



tJ: 33. A video delivery architecture as recited in claim 3 1 , wherein the client 

reconstructs an enhancement layer in a particular frame from an enhancement layer of a 
reference reconstructed frame. 
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34. A method for coding macroblock video data according to layered coding 
techniques in which the macroblock video data is represented as multi-layered frames, each 
frame having a plurality of references in multiple layers ranging from a base layer of low 
quality to enhancement layers of increasingly higher quality, the method comprising: 
encoding macroblocks to produce a first bitstream representing a base layer; 
encoding macroblocks to produce a second bitstream representing a plurality of high 
resolution enhancement layers and a plurality of low resolution enhancement layers using an 
INTER coding mode selected from the group consisting of: 

a first coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 
both the low and high resolution components are predicted in the 
temporal domain from a low quality, low resolution reference in a previous 
frame; and 

a high resolution, high quality reference in the current frame is 

reconstructed from the low quality, low resolution reference in the previous 
frame; 

a second coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 

the low resolution components are predicted in the temporal domain 
from a low quality, low resolution reference in a previous frame; 

the high resolution components are predicted in the temporal domain 
from a high quality, high resolution reference of the previous frame; and 
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a high resolution, high quality reference in the current frame is 
reconstructed from, and is of the same quality as, the high quality, high 
resolution reference in the previous frame; 

a third coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 

one said low resolution enhancement layer predicts another said low 
resolution enhancement layer of higher quality in the transform domain; 

one said high resolution enhancement layer predicts another said high 
resolution enhancement layer of higher quality in the transform domain; 

one said low resolution enhancement layer predicts in the transform 
domain one said high resolution enhancement layer; 

a high resolution, high quality reference in the current frame is 
reconstructed from the low quality, low resolution reference in the previous 
frame; 

a fourth coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 
the one or more enhancement layers include a plurality of high 
resolution enhancement layers and a plurality of low resolution enhancement 
layers; 

one said low resolution enhancement layer predicts another said low 
resolution enhancement layer of higher quality in the transform domain; 

one said high resolution enhancement layer predicts another said high 
resolution enhancement layer of higher quality in the transform domain; 
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the high resolution enhancement layers are not predicted in the 
transform domain from the low resolution enhancement layers; 

the low resolution components are predicted in the temporal domain 
from a low quality, low resolution reference in a previous frame; 

the high resolution components are predicted in the temporal domain 
from a high quality, high resolution reference of the previous frame; and 

a high resolution, high quality reference in the current frame is 
reconstructed from, and is of the same quality as, the high quality, high 
resolution reference in the previous frame; 

a fifth coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 
the one or more enhancement layers include a plurality of high 
resolution enhancement layers and a plurality of low resolution enhancement 
layers; 

one said low resolution enhancement layer predicts another said low 
resolution enhancement layer of higher quality in the transform domain; 

one said high resolution enhancement layer predicts another said high 
resolution enhancement layer of higher quality in the transform domain; 

the high resolution enhancement layers are not predicted in the 
transform domain from the low resolution enhancement layers; 

the low resolution components are predicted in the temporal domain 
from a low quality, low resolution reference in a previous frame; 

the high resolution components are predicted in the temporal domain 
from a high quality, high resolution reference of the previous frame; and 
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a high resolution, high quahty reference in a current frame is 
reconstructed from the low quality, low resolution reference in the previous 
frame; 

a sixth coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 

both of the low and high resolution components are predicted in the 
temporal domain in the temporal domain from a high quality, high resolution 
reference in a previous frame; and 

a low resolution, low quality reference in the current frame is 
reconstructed from a low quality, low resolution reference in the previous 
frame; 

a seventh coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in a current frame, wherein: 
both of the low and high resolution components are predicted in the 
temporal domain from a high quality, high resolution reference in a previous 
frame; and 

a high resolution, high quality reference in the current frame is 
reconstructed from the high quality, high resolution reference in the previous 
frame. 

35. A computer-readable medium having computer-executable instructions, 
which when executed on a processor, direct a computer to perform the steps of Claim 34. 
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36. The method as defined in Claim 34, wherein the layered coding techniques 
comprise a Progressive Fine Granularity Scalable (PFGS) video coding. 

37. A method for coding macroblock video data according to layered coding 
techniques in which the macroblock video data is represented as multi-layered frames in 
multiple layers ranging from a base layer of low quality to enhancement layers of 
increasingly higher quality, the frames including frames «-7, «, andn+1, wherein frames n-l 
and «+i have a plurality of references, the method comprising: 

encoding macroblocks to produce a first bitstream representing a base layer; 
encoding macroblocks to produce a second bitstream representing a plurality of high 

resolution enhancement layers and a plurality of low resolution enhancement layers using an 

INTER coding mode selected from the group consisting of: 

a first coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in the n frame, wherein both 
the low and high resolution components in the n frame are predicted in the temporal 
domain firom a low quality, low resolution reference in each of the n-l and w+7 
frames; 

a second coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components for the n frame by temporal 
predictions from: 



a low quality, low resolution reference in each of the n-1 and 



frames to produce low quality, low resolution components in n firame; and 



a high quality, high resolution reference in each of the n-1 and n+7 



frames to produce high quality, high resolution components in n frame; 
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wherein one said low resolution, low quality component in the n frame is 
used for prediction in the transform domain of one said high quality, high resolution 
component in n frame; 

a third coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components for the n frame by temporal 
predictions from: 

a low quality, low resolution reference in each of the n-I and «+/ 

frames to produce low quality, low resolution components in n frame; and 
a high quality, high resolution reference in each of the n-1 and «+7 

frames to produce high quality, high resolution components in n frame; 

wherein the low resolution, low quality components in the n frame are not 
used for prediction in the transform domain of the high quality, high resolution 
components in n frame. 

38. A computer-readable medium having computer-executable instructions, 
which when executed on a processor, direct a computer to perform the steps of Claim 37. 

39. The method as defined in Claim 37, wherein the layered coding techniques 
comprise a Progressive Fine Granularity Scalable (PFGS) video coding. 

40. A video delivery architecture, comprising: 

a content provider having a video storage to store video data, a video server to serve 
the video data over a network, a base layer encoder, and an enhancement layer encoder, the 
video server being configured to encode macroblocks of video data according to layered 
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coding techniques in which the macroblock video data is represented as multi-layered 
frames, each frame having a plurality of references in multiple layers ranging from a base 
layer of low quality to enhancement layers of increasingly higher quality, the frames 
including frames n-l, n, and n-^l, wherein frames n-l and n+1 have a plurality of 
references, wherein: 

the base layer encoder encodes macroblocks to produce a first bitstream 
representing a base layer; and 

the enhancement layer encoder encodes macroblocks to produce a second 
bitstream representing a plurality of high resolution enhancement layers and a 
plurality of low resolution enhancement layers using an INTER coding mode 
selected from the group consisting of: 

a first coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components in the n frame, wherein 
both the low and high resolution components in the n frame are predicted in 
the temporal domain from a low quality, low resolution reference in each of 
the n-1 and n+7 frames; 

a second coding mode that encodes macroblocks to produce the 
second bitstream having high and low resolution components for the n frame 
by temporal predictions from: 

a low quality, low resolution reference in each of the n-1 and 
n+1 frames to produce low quality, low resolution components in n 
frame; and 
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a high quality, high resolution reference in each of the n-l and 
n+l frames to produce high quality, high resolution components in n 
frame; 

wherein one said low resolution, low quality component in the n 
frame is used for prediction in the transform domain of one said high quality, 
high resolution component in n frame; 

a third coding mode that encodes macroblocks to produce the second 
bitstream having high and low resolution components for the n frame by 
temporal predictions from: 

a low quality, low resolution reference in each of the n-l and 

«+7 frames to produce low quality, low resolution components in n 

frame; and 

a high quality, high resolution reference in each of the n-1 and 
«+7 frames to produce high quality, high resolution components in n 
frame; 

wherein the low resolution, low quality components in the n frame are not 
used for prediction in the transform domain of the high quality, high 
resolution components in n frame; 
a client configured to receive the encoded video data served from the content 
provider, the client being configured to decode the video data. 



41 . A video delivery architecture as recited in claim 40, wherein: 
the video server transmits the encoded video data as composing the base layer and the 
one or more of the enhancement layers; and 
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the client decodes the video data from the base layer and the one or more enhancement 
layers. 

42. A video delivery architecture as recited in claim 40, wherein the client 
reconstructs an enhancement layer in a particular frame from an enhancement layer of a 
reference reconstructed frame. 
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